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•ABSTRACT 



A randomization model appropriate for evaluating priority effects * 
in fr^e recall (i.e., whejiher "n^w" items are recalled\ p3;-ior to "old" 
Items') . is discussed- and. related to well-known nonparametric signif- 
icance tests. Since the bases for the measures that have been suggested 
in the psychological literature may be interpreted either in terms of 
Wilccu^n's raniP sum statistic or through a speci^c ^entry in a' 2 x 2 _ 
contingency table, alternative ^ indices of priority can be adopted di- 
rectly f ran. classical nonparametrie statistics. Finally, tlie mean, and 
variance formulas for* a general correlational statistic are provided 



that specialize to the momeots for the two measures alrealcly in common 
use. 



* 1 



Vll 



\ *" ' . ABSTRACT ' ' . . • . . 

A randomization model appropriate for evaluating priority effects * 
in fr^e recall (i.e.^ whejiher' "n^w" items are recalled, p^ior to "old" 
Items'), is discussed' and related to well-known nonparametric signif- 
icance tests. Since the bases for the measures that have been suggested 
in the psychological literature may be interpreted either in t'erms of 
Wilccu^n's rani^sura statistic or through a speci^c entry in a~ 2 x 2 _ 
contingency table, alternative -indices of priority can be adopted di- 

rectly from. classical nonparametrie statistics. Finally, tlie mean, and 

• • • « 

variance formulas for' a general correlational statistic are provided 



that specialize to the moments for the two measures alrealfly in common 
use. 



INTRbDUCTJOfi ; 

Most of the quant^itative literature that has dealt with the 
^free-recall Je^ftincf paradigm has emphasized the* measurement problems 
encountered In -identifying category "clustering^ within a subject's 
protocol (see Shuell, ;969) . Althbuglj this interes^t may be due in • 
part to historical precedents, it is still* surprising that a related 
and well-known {)henomenon from the 3^me experimental literature, called ' 
the "priority effect" (see Postman, 1972), has been virtually . ignored 
by methodologists. The onlM relevant paper appears to be the recent 
^contribution by Flores and Brown (1974). To be more specifioy suppose, 
that a subject is required to learn a list of item.s 6ver several trials r 
Priority ds evidenced whenever the subject recalls a "new" item on a 
particular trial (i.e., an item emitted for the "firsi: tijne) prior tb 
recalling a,n "oid" item (i.e., an item already recalled oh one or more 
previous trials).. Apparently, no genera l74«ethodo logy has been proposed 
'for .assessing priority effects cbmpaurable to what now exists for evaluat 
* ing category clustering, even though exactly the same class of random- 
ization proce<piures is appropriate in both instances. ^ 

It*is^ not the intent' of this.papjer to develop any ^radically new 
inference models fdt assessing^ priority/effectsjf in fact, the discussion 
merely illustrates how randomization cpncepts that are very familiar in 
nonparametric statistics may be applied to this rather specific experi- 
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mental paradi^ of interest to psychologists working in the axea of 
meitKyry organization.- X)nce tfe general statistical framework has«been 

d^velope^. the two measures t5f the^ priorityTphenomenon that have been 

» * \ • ' 

suggested by. other authors can be related directly to comipon nonpara- 
metric sJlgpif icanqe te*ts. ,More importantly,. thi& relationship sug-. 
gests possible ext^nsion#^ of the' basic conc:ept pf priority to include ^ 
alternative experimental hypotheses that involve more than a** simple 
zero-one, (i.e., "old" - "new") catf gorizatloji of thCfl^ms. For irt- * 

stajice^ conilpts that are mea^Lrable. on a more comprehensijVe scale, 

/ s 

such' as meaningfulness, frequency/ concreteness , ,and ^he like, can 

be evaluated in essentially the same manner, as long as the. ^appropriate 

. ' / - ' ' ' ^ \ 

'Statistical generalizations are developed 'at the. outset. ' . 

Finally, the *n©nparametric connections provide several formal 

indices of priorlt^^-espec^^lly for'the usuajl diehotcmous application — 

that have the same type of operational interpretation- as advocate by 

Goodman and Kriiskal (1954)^ iJptheir classic ,papet on measuring associr. 

ation xn a contingency table. \ Since the GoodmanrKruskar"arguments for'^ 

a "good" -measure of associatidn are rather well-accepted in 'psychology 

(se^'Hays, 1973, Qhapters 17 artd 18), it, is of* interest to point out 

the availability of indices of |>riority that have similar operational 

signirican(je. ^ ■ • . \ 



0 * 
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II 

BACKGROUND 



^ As a vf§y of formalizing tlie problem of measurikg priority ef- 
fects, s\appos^ that on a particular ttial' ^ subject recall^ n^Vtems, 

o ,o ,...,o , in, the order indicate(?. In other words^ o is^irepaaied 

1 2 n ' « ' • ' ^ — — ^ 

first, o is recalled' second, and so on. Furthermore, ea«h of the 

' 2 ' - ' - « 

objects,' o., has 'an associated numerical value, X/, that den^otes ^ome 



characteristic of the item. In the most common application 



if was recalled on a pre-, 



vious 



trial (aJ "old" item) 



(1) ' / . • x/= 



1 



if o^ was not recalled on a 
previous trial (a. "new" item). 
Finaljy, a second Varicible y. is at^tached to o^ that indicates tat^e 
relative position Tof the object in the recall sequence; for inst^ce> 

^ . as one important case to^^ dC^fiftiUer^ explicitly r we us^y^ to denot^ 

~ ' ' ' ' ' ' \ \ 

■ . . > ' " . - ' : 

(2) . . y. = i. ^ ^ . 

For convenieftc^, y. defined^s in (2) will be called a rank function . 

. t 1 ' < 4 

In, summary, when the variable x^ is paired with the corresponding ^ 
^ variable y^, 'evidence for k priority effect exists if the x^'s thatj 
: are I's tend to be paired with the y^'s that are small. (Conversely, t 



"negative" priority exists if the x^'s that are 1 "is, tend to be paired 
« • 

with the y.'s that are ISrgeJ * • . . 

1 . -i- . ^ 

^Since a' sequence " of n bivaiciat^ * observations is available using 
the x.'s and 'the y.'s, it is natural to consider some type of^ correla- 
tional index for*a measure of relationship. As one simple ^>osslbility r 
a raw (or unnormalized) index of the priority effect iS obtained by 
the product-moment statistic - . * ' 

n ^ / ' 

r = Z x.y.. * . 

Given the specifications of and y^ in (1) and J2) , res^^ectively, 
•r is the Slim of the. recall ranks for all of. the new items, a^d after 

a suitably normal ijzation, fopns the basis of a measure IntrcJduced by 
^ Battig, Allen and' Jensen Xl^^S) • -Numerous alternatives for a f^nalnor 

malization exi^t/ 'however/ cuid^consequently/. ,for convenience in the 

initial discussion, onl^ the index T will be considered directly. 



The probtem' of obtaining an appropriate n6rmaIizatij.on will be dealt / 
with in a later section. 

Using the genera* forAi of the fetatistic T, otheJr possible measures 

K * V » 

of priority can be developed merely by varying* the definition of y.« 

V ■ ■ ■ A . ^* 

As a .second important' illustr\ati©ny suppose no longer denotes, rank 

position, in ^ecrall. Instead, \the protocol is ^ost dl»h\)tomized af 

' I th 
*some point, say ^t the It- teca\^r position, ^and y^ is given by: 

\ if i > R; . 



\ 

\ 



Tor* convenl*cVyv,yr^e?-^ is called «a. 4ichot;proy function ^' ^ 

Ifv the', simf^ie d^finilrion iru^d) fpr x^. is still appropriate, but'yt 
is now*a diohotomy function, T is the fiumber of new. xten\s. in the 



first R position^ of the 'prcitocoj.^ Furtdermore, F correspon<js- toj an • ^ 
unnormaliz^d version of a measure originally suggested hy Postman an(j \ 
.Keppel (1968) -and Shuell and Keppel ?1968) ."^ .J .^|^ 

In^sfiort, measures that ar^ based on different scoring functions 

for y. lead' to differen^. statistics that may have va^lue in -assessing 

^ > * * J ' ' ' ^ . ' 

•priority effects, As^ f Ather g^erali'zation t>hat\was mentioned ' ^ 

earlier .""JChe variable x.' itself can be extended beyopd^the simple ' ^. 

zeife-one function considered ^.n (1) cua^ F sti\l'l:an lp6 used as a 

measure of a monotonic relationship between recall^ position amd the 

information regarding the >9e6afiLled iteins provided by x\ . , Since F is* 

the' ciucial quantity no mfiftter what ^peoif ic definitions ^ij'e ultimately 

selected, tfie next section dJ^cusses ^ ramdonjizattion distribution for 

in some ^ietail. Al^so, the applications of this randomization d is-? 
. ' ' ' * *" * 

*t'ribution for, the two special cases discussed in the- literature, defined^ 

by (1) for X. and either (2) or,(G) for y. , will be pointed out explic- 
1* wl'^ #• 

itly. ' — 

' Althailgh no s.tatistic^pil inference* model has been .discussed as yet, St 
at least one argument is already apparent for preferrin<^ the rank ^unc* ; 

tion definition for y, over the dichotomy function. The dichotomy 

K . ^ .. ■. . 

function uses less iuformati^oa froih the protocbl, and evlmon a priori 



1 ) • ' * 

'The index F has been keyed in such a way ^that small va*iues indicate 

a priority effect fcr the ranJc function but Ihrge - values indicate a 

pr:iority effect for the dichotomy function^* Altshdugl^. this discrep*an5?V 

is somewhat of an* inconvenience, ij/ paral^lels the , aval Idbl^ 'literature. 



grounds, .would Y^aye 'to be considered 'less sensitiv,e. Moreover/ the 
loss -of protocol informatiQ^ ^that is Vbuilt into" the dichotomy func 
tion is not justified- by .any later statistical ad\fentage, either cpn 
ceptualiy -or' computationaliy. • In fact, the rank^^jJfcdfc itself is 
*so?n^hl^t ' easier to dev^Blop, and in" spveral instianceS will actuall/ 
reguXt sin simpler* fprmuj^as. , • * • 



RANDOMIZATION PROCElKiRES TOR r 



Although the val-ue of t-^e index T can be calculated with ease ^ 
an inference problem stillexists in decidj.ng Aether the' size, of V 
i^Pl^fficien'tly extreme *t±> reject a "null" .hypbthesis of no priority 
effect. ' One bonveni^int technique for modeling this 'jiiull" hypothesis 
is through the concept" of a permtitatioi\ distributipn well known in 
ttonparametric statistics. In particular, it is^pssumed that all n! x 
possible ways of* assigning the x^'s to the fixed seque'hce o^ the y^'s 
are equally likely' a priori . Furthermore, for^ail n! possible alloca- 
tions of th^ x.'s,. the values of V are obtained ' and ^tjjSfen t^led to 
form a frequency distribution for which is then^ treated' as its^ 
"null" probability distribution. 

' As a simple example that may heip to »clarif / the inference proceSs> 
suppose ^at n = 4, with 2 new items emitted by the ^Ubject ih the 

first and second positions or his protocol. Usirig the definitions 

• . • \ 

for X. and y. in <1) and (2), respectively, we ^jve x = 1, x = 1, 

i is T . ^ 

= 0, = 0; and =• 1,' = 2, y^ = 3, = 4. .Assuming that 
the y 's are fixed, the 41 = 24 possible orderings of the x^'« and 

* • • ' • 

' the associatej^' values o/ V are as follows (for later purposes, the * 
last column- ^Iso presents the index T based on the definition for y^ 
given in (3) for R « J.) : 
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8' . 



"A 



\ 




Permutation 



^1 ^2 ^3 ^4 



^1 ^2 ^4 "3 



^1 ''3 ^2 *4 



^1 .^3 ^^4 ,^2 

4 xy X3 

^1 v/3 ^2 
^^^1 ^3 ^4 

*2 ^1«*^4 ^3 

^2^^ ^4 



^2 ^3 ^4 ^1 
x^ x^ x^ x^ 



.X3 x^ X2 X4 



X3 x^ 
^3 ^2 ^1. ^ 



^3 ^2 ^4 ^1 



3 4 J 



X3 x^ x^ 



^4 ''1 ^2 ^3 



X3 x^ 



^4 ^2 '^l ^3 



"4 .^2 *3 ^1 



x^ X3 x^ x^ 



x^ X3 x^ 5Cj^ 



r (Rank Function) 
3 

••3 

.4 

5 • • ' 

5 
3 

.3 

» 4 • 

5 

5 . 
6 
5 

6 - 
7 

7 - 
5 
6 



r (TDicho^roy Function)' 
1 
1 



r 



■\ 



5- : 



\ 



7 
.7 
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1.- 

1 

1 

1 

1 

1 

i ' 
1 
1 
1 

0, 



0 
0 

0 
0 
0 
0 
0 
0 



For the xrank^ function (secdnfi colt^) , these 24 ortieringsr provide the 
probability distribution* given below, which now can be used to evaluate 
the given protocol 'llndeJ^ the hypothesis of no priority effect. 



r 


• ^Frequency 


Probability 


3 




' 1/6 


4 




1/6' 




. . , ^ ■ 8 •/ 


2/6 


6 




4 f 

1/6 


7 




' 1/6" 




/ . 24 * . 


•1-0 



In our example, the* observed value of V ^^obs^ ^^ ^' consequently, 
the hypothesis of'* nb priority effect can be* re>e<5ted at a significance 
level of V24 ='a67,A,e., Pt^I^oj^g 1^)/^ .167]^ 

The readeV may hr^v^ already recognized the similarity between 
this example atid a very simple- applicatioh^of Wilqoxon's two-sampie 
rink sum test di^ussed in many elementary statistics texts. More 
Wplicitly'r if ic. '•is defined as in (1) and y. by. the rank function, 
then^ the permutation distribution for T i3 equivalent to the exact 
sapling distribution of Wilcoxon's te^t statistic used for xfi^nxinq 
two independent samples. One sample is defined J>y the "newV i,^ems, 
the second sample i^ defined by the "old" items-, and -the -dependent . 
variable is the recall rank of *a particular item within the protbco],. 
Consec^ently, in <>ur illustration,, two groups are formed with two ob- 

It should be noted that ^he Mann-Whitney ,U statistic . prpvid^s a statis- 
tically, equivalent version of this test. Also, in some special cases 
. other cbmmoxi randomization' tests 'may be valuable; for instanpe, when 
the "old" and "new" items can.be matqhed on an a priori basis, sijrv- 
tests or ohe-saAipre Wilcoxon tests migljt be wp^-th consider itig. ? -\ 



serrations in eacl\, with outcomes as follows: 



Group I ("New" items) I 


Group II ("Old" items) 


1- 
2 


, . 3 ■ ; V 



Since T is merely the sum of the ranks in Group I and this statistic 

\ • 

the basis of Wilcoxon's test, the exaqt permutation distribution i^s 

t^led in nitrous spurges (for instance, see Bradley, 1968, pp.' 105-^ 

117). When 'viewed from this perspective, la simplification of the. enuro— 

V, ' ^ » 

erated permutation distribution becomes apparent, i.e., if we have I 

/"new" items and n-I "old" items, then only {) = ( possible .rank - 
^ I n-'X 

4 « 

sums need be considered. Here,, {^) =6, ahd the sum of 3 for the "new" 
items provides a probability of 1/^, = .167. ' '* 

When y. is the. dichotomyy^nction and x. is 'defined- as in (1), a 
similar equivalence caprbe developed between the permutation distribution 

for r an4 anothej<^ommon nonpar aiqe trie test.' I'he permutation di^tri- 

> • \ ' ' ' 

bution in><?nis ca3e is the same as the distribution of b in the foliow- 

X 2 contiug^icy tabl^e with fixed marginals: 



Position in protocol 





R 






"New^* items 


b 


l-b 


I 


"Old" items 


R-b 


n-R-I+b 

f 


n-I 




R 


' n-R 


n 



The distribution of b is hypergepmetric and may be' found explicitly ^ ' 

R n"~R n * 
•by -SCr =s b) = (^) (^ ,.)/(^) as. in the Irwin-Fisher <or Fisher efxact) \ 
b I-b I , ^ 

test. Thus, the distribution of T for our single example (i.e.., the 



' peo^utation l^iLsting) ^ould hCave been obtained by 



r = 0J|= (J)(^)>(^) =3/6=1/2;^" 
[(.r.= li ^A])(bf/{t\ = 3/6 = 1/2, 



arid the p-vaj.u| associated with the Sbserved value of T^. = l in our 

, ]■ j- ^ <% - ■ .""^^ 

simple example is 1/2. Again/, nvore general tafbles for the exact per- 

r ' / \ • » ' ' • 

mutation distribution of 'r are avAlable* merely by reinterpreting tl^e 
inference problem as one-^ of -evaluating association, in a 2 x 2 contiri- 
gency table (je.g. see Bradley, 1968, ppf, 193-203), 

AlthbMgh an ^'^plication. of Wilcoxon's and Fish^'s exact tests' 
could, l^ad to' different result §f^^ for the same protocol, obviously some 
generaly^consistency has to be present. In fact, David lancj Barton 
(1^62, p. 190) state that the joint distribution of the T's based on 
the definitions of y^ in, (2) ai^d (3) when x^ is zero-one i& approxi- 
'mately bivariate normal with an exact coryelat^oh of 



3R(n-R) 



(n-l) (n+1) 



Surprisingly, this correlation depends onl^ on the point of dichotomy 



R and not on thenyimber of new items 'actually present. When R Is close 
to %he median object, -this correlation is close to its maxiinuiti absolute 
value. Thus,, the T statistics base^^n the two definitions of y^ are 
most ^consistent "when R divides the protocol .approximately in half. 
For instance, ia^our previous exapnple with n ~ 4: For R = 1^ the 
correlation between the tvo given statistics would be -,78; if R = 2 

were chosen instead, the correlation would be -,89. For any value of 

♦ > • 

l{, 'a large-scuhple Ci.e^, n ®) correlation can be computed merely by 



substituting in the above formula, • . " 

Wh'en is a general numerical variable that does not have a 
simple dichotcanous structure that would allow the use of published 
tablets for the; corresponding permutation distributisen, other procedures 

must be Aised in testing sigfiif icahce. JSeveral strategies 'are possible, 

' *' ^ . . » - ' • ' • 

with the. most obvious Being to rely on a large-'san^^le nonwl approxi- 

mation based on the fallowing formulas for the mean and^ varian^ce of 

r (see .Kendall,^ 1970, 'p. 7t^) : . ' .' 



• 



' , E(r) = (l/n) ( Z X.) ( Z y.); 

1=1 1=^1 . • " 

: ' ' ^ ' ' . ^ 

Var(r) = (l/(n-I))( Z^(x,-x)^)( Z (y.-y) ). 

i=l ^ i=l ^ 

For'y, = i arid x, defined as, in (1), these expressions reduce to the •' 

well-known moments for " Wilcoxon' s two-sanqjle statistic: * 

E(r^ = I(n+i)/2; . . ■ • ' 

4 • ■ . Var(r>'= (n+l)I(n-I)/12. - ' . 

**. 

Also, when x. is defined as ,in (1) and"^, as in (3)', we have 

ECT) = kl/n'; ' ' . • ^ 

" - ' ■ ' ' .' ' 2 ' • ' ' 

Var(r) = (n-I)IRtn-R)/(n (n-1) ) . - 

_ \ • ^. 

; In addition to' the^ large-san^le ^^eat ^hat wOuld compare * 

Z = (r-E(r) )//var (D to tige standard normal distribution, a cqnser- 

vative inference. strategy could b^ obtained by applying- Chebyschevi 9 ' 
"* > ^ * * ' . . 

inequality and^ merely stating thart the "ti;jie" significance level can 

be no larger' than lA^. (For the natural one-tailed test, Cantelli's 

inequality would assure us that the "true" signi£icance level can be 

i4 Urger than I/(Z^^l).f \0r possibly, fqllowin^Hope ^ (1968) ^ more. 



- - j' 

'\ ' „ ' . - ^ . ^ 

accurate significance 'levels- could be found by generating* a san5)J.e . 

' . ^ ^ ' . ' . - ' >> 

of tl^e r stati3tics under the randomness hypothesis and .using, these 

data to approximate the exact permutation distribution. Further *dis- 

cus'sions of the^e latter alternatives for. several related problems;* 

* * * ■■ 

are given in Hubert & Levin (1976; .in press, a) b). 
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F6RMAL INDICES OF/RIORHY . ' - 

The previous section 'has suggested a randomization strategy for 

evaluating ^t he relative size* of T, and, in particular, for determining, 

whether T is sufficiently extreme to reject a -hypothesis that the 

given protocol actually resulted frcan a raindom permutation of the 

x.'s. Obviously, sinc^. general mean and variance expressions for F 
1 * * ^ - , , 

^re ayailaSle, a: large rlCunber of possible normalizations could be en- 
tertained; In fact, every index that has been suggested for categor- 
ical clustering could be considered here as well.^ For purposes of 

connecting , the ^present discussion to the* literature on categorical 

• ' it 

clustering, two saui^le indices are J)resented, one, discussed previously 

by Flores and Brown (1974) called thp ^^Relktive Index of 'Priority (RIP) 

' and the second defined by the Z score* (ised in the larCje;^ai4>le hypoth- 

esis test. We leave it to the feader to, develop other more traditional 

carwlidates by generalizing^ the index list given in Hubert and Levin 

■ * . ■ • ] ■ ^' 

(in press a) ,that summarizes passible indices of »arteggri^l clusteripg. 



For a general index F, the Flores and Blrown RIP formula may be 
d^fi^ned as: • . • 

RIP = JF - «(F)J/tF ^ - E(r)]. 

„ ^ / "* max ' 

The "F'' " term is obtained by calculat;ing T for a protocol in whicli 
the .given x^'s <ir e order ed from smallest to largest. Flores and Bv^im 
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airgue. the merits of the KIP formula, and in so doing, repeat m^ny of 
the same justifications -for using an emaiogqus index within the Cat-^ ' 
egorical clustering contej^t <but see Sternberg & Tulving^ l¥76, 
a discussion of problems associated with this index?. * 

• Tl>e second ind-ex is the Z score ^or P based on the general expec-' 
tation and variance for^iulas: , ^ - _ • 

z= [I* -"E(r)]//v^r(r). * ^ ' 

Interestingly/ this Z value has a very simple relationship to th^ 
.product moment correlation between^x, and y. : 



Z =y(n-l) r^^. • ; . 

In other words, the Z score is proporti^onal to the correlation between, 
and y^; and furthermore, the correlation coeffi-cient per sq coilld be 
.considered as a c<^petitor" to 2 as an index of priority. Since 
ECr^ ) = Or r is "corrected" for cha^>ce^ in ^ natural ^Jay; also, 

» AI Xl 

r has an especially siirple vpermutation variance of l/(n-i). 

^ 'As mentioned previously, instead of merely adopjting indices of 
priority from^^tlji^ ca^egoricai clustering field, other alierhatiges 
can be*, found in the statistical literature that have operatic>nal meem- 

^^^^ w ^ 

ing (i.e., that hav^ direct interpretatfB^ in terms of probability 
statements regarding the. given protocol). For in^^nce,, using a zero- 
one function X^, and y^ given by '(2), Wilcoxon's two-san^l^test ' st^a- 
tistic results; consequently, a particularly* interesting measure of 
prior ij:y is given by , ^ ^ 

. p = U/(I(n-I)),' ^ ' , 

where * * ' ^ , * 

U = [(n-I) (n+I+l)/«ft - r (the Mann-Whitney U 
* . statistic). 
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. S^cifically, p can be interpreted wi£h respect t;p the .given prbtocol 
as ..follows: If a "new" and an >'*^oia" item are selected at random from 
the"'protocol,- then p is the prcrt^ability that the "new" item appears . ^ 

A' 

Jb^fjDT^ the "old" item in the^ given protoeol. Since* U is obtaine^i ^ . 
frcxn r by simple linear transformation that jinvolves only known 

quantities, the permutation distribution for p can be found easily * ^' 

by using the dlstributiofi for In short, since p has a probabi- 
lis tic interpretation with respect' to the 9iv|^ sample data, it is • ' 
s unnecessarv to define an unknown population parameter (or population'- 
analogue of The sample index) 'to justify using the statistic. Similar' 
pBobabilf^tits arguments for several other measyires of association are 



discussed inY^tail by Hays (1973, Chapters 17 and 18), e.g., for 
Kendall's x and Goodman-Kruskal 's^ A and y. 

As a second illustration usibg^the dichotomy definition for y. 
given in (3) that leads, to the 2x2 contingency table framework, one 
probabilistic measure of .f^riority ma^ be defined through • 

^ ' - - P = b(n-R-I+b)- , ^ 

^ [b(n-R-I+b; + (R-b)(I-b)] 

and given the following interpretation: ' Suppose two items are drawn 

at random from ^the protocol with one iteto being "new" and the other 

."^"old," and moreover, belo/iging to different sections of the dichoto- ' 

mized protocol; then, p is the probability, with respect to the ^given 

, ^ ^ V — 

protocol, that the "new" item is in the first part of the protocol and 
^he "old" item in the second part. Again, the permutation distribution 

/ ' ' A 

for r leads* directly to a permutatio'ti distribution for p, but in this 
ffase the form of the transformafion between p and b is somewhat more 
\^omplex than before. . • 

23 • 



^ OTHER GENERALIZATIONS ' \ 



Although most af the previous discussion hag emphasized proce-* 



dures that would be appropriate when ;x/ is dichot.omous, the general 

index T was defined for aind can be used when x. characterizes a nu-* 

^ \ ' 

merical variable with mOre than two distinqt values. In addition to 
relying on the di,rect correlation index T, however, ^ainother approach 



• is suggested by traditional nonparametric statistics \if we assume that 

jthe, say, K distinct values of x., actually label K classes within a , 

' ' ^ %. • 

traditional one-way analysis-of-varianc^ framework,. The observations 

within a particular category would then correspond -to either the re- . ' 

call ranks for those items with a specific numerical label or 0-.1 ranks 

for the portion of the protocql in which th^ item is located,, , Fpr* the 

former case in which recall rank is used' as a dependent variaQ^le, And _ 

if x^ 4^notes labels that are not ordered , the appropriate nonpara-^ ^ ' 

metric generalizations wo^ld be to either the Krugkal-Wallis one-way 

analysi^^^l^-varianpe paradigrh, or to, a formulation as a contingency ' 

table with one ordered and one unordered fat:tor (see Hubert, 197^, 

If x^ denotr^s l^bfels that order the categories, the agpropriate non- 

^ parametric generalizations* w^ld" be to am ordTered-ti^ategory analysis- 

of-var;Lance pax?adigm commonly used to te^for a monotonic .trend in- a 

Kruskal-Wal^is framework, e.g. , see Hollander and Wolfe (r973, pp. 120^ 

^ * 

* • • • » ' 

123) and Marascuilo and McSweeney (1967), For the 0-1 ^dependent ♦ 
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measure, the inference problem reduces to'anievafuation of a K by 2 
contingency table, with ordered classes, and jxjssibly, a measure such 
as Goodmah-Kruskal ' s y< could be considered. In fact, siijce the ap- 
•propriate mean and vari^ance parauneters are available for all of ,these 
"^neralizations (for instance, ^see David cind Barton, 1962*) ,/ the ^cir- 
ious measu!res used for categorical clustering could t>e, adopted here 
as well, . , ^ • " ' 

I 
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